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ABSTRACT 

The CITARS task design and objectives are reviewed and final results . 
are presented, together with conclusions and recomnendations . / It was found 
that several factors had a significant effect on crop identification per- 
formance: (a) crop maturity and site characteristics, (to) which of several 

.different single-date automatic data processing procedures was used for 
local recognition, (c) nonlocal recognition, both with and without prepro- 
cessing for the extension of recognition signatures, and (d) use of multi- 
date (multitemporal) data* It also was found that classification accuracy 
for field center pixels was not a reliable indicator of proportion estimation 
performance for whole areas, that bias was present in proportion estimates, 
and that training data and procedures strongly influenced crop identification 
performance . 


1. INTRODUCTION AND OBJECTIVES 

CITARS (Crop Identification Technology Assessment for Remote Sensing) was a joint task to 
quantify the crop identification performance (CIP) achievable with several automatic data process- 
ing (ADP) techniques operating on remote sensor data. It was conducted from April 1973 to April 
1975. Participants were the Earth Observations Division (EOD) of the Johnson Space Center, the 
Environmental Research Institute of Michigan (ERIM) , the Laboratory for Applications of Remote 
Sensing (LARS) of Purdue University, and the Agricultural Stabilization and Conservation Service 
(ASCS) of the U.S, Department of Agriculture. The CITARS task design was presented in Ref. 1 which 
also includes objectives, analysis methodology, experimental procedures, description of ADP pro- 
cedures, and first results of CITARS. A more extensive description and documentation of the entire 
project can be found in Ref. 2, The remainder of this section presents a brief overview of the . 
CITARS task design and objectives. 

7; ; The principal assessments made were of crop identification performance for corn and soybeans 
in six sites in Illinois and Indiana. The remote sensor data analyzed were collected by the ERTS-1 
(now called LANDSAfui) multispectral scanner (MSS) periodically throughout the 1973 growing season. 
The ADP procedures corrpared were predefined at EOD, ERIM, and LARS in such a way as to minimize 
subjective analyst judgment and Intervention in the crop identification process . 

Specific objectives of CITARS included performance comparisons to answer the following 
questions: - .. yj" ‘ \'.Z 


* Proceedings of the Tenth International Symposium on Remote Sensing of 
Environment, Ann Arbor, Michigan, October 6-10, 1975. 



1. How does corn and soybean identification performance vary with time during the growing 
season? 

2. How does crop identification performance vary among different geographic locations 
having different soils, weather, management practices, crop distributions, and field 
sizes? 

3. How much variation in crop Identification performance is observed among the different 
data processing procedures? 

t . Can signal statistics acquired from one time or location be used to identify crops 
at other locations and/or tines? 

5. Does use of radiometric preprocessing extend the use of training statistics and/or 
increase nonlocal crop identification performance? 

6. Does use of nvrltitemporal data increase crop identification performance? 

To accomplish. the objectives, five major tasks had to be completed. These were: (1) acqui- 

sition and preparation of an ERTS data set with ancillary data sufficient to support the experi- 
mental objectives and design, (2) computer-aided processing of this data set with the selected 
classification algorithms and procedures, (3) quantification of the crop identification perfor- 
mances in a manner which would permit a quantitative evaluation of the ability of these procedures 
to satisfy agricultural applications requirements, (it), a statistical analysis to quantitatively 
evaluate the impact of major factors known to affect crop identification performance, and (5) an 
interpretation of the results to ascertain the underlying physical factors responsible for the 
results, to draw inferences as to the status of the technology as it relates to agricultural 
applications, and to make recommendations as to where the technology must be strengthened. 


2, KEY TECHNICAL ACCQWLISHMENTS 

As reported earlier [1] an ERTS data set with supporting ancillary data was acquired and pre- 
pared which met the requirements of the CITARS experimental design, except for completeness of 
ERTS coverage. Assembly of the data set included: (1) acquisition of crop identification and 

other agronomic ground truth data by ASCS, (2) acquisition and interpretation of color infrared 
aerial photography to extend the field identification data acquired by ASCS to additional fields, 
(3) registration and geometric correction of multi temporal ERTS MSS data for the test segments, 
and (4) location of field and section coordinates in the ERTS data. In addition, repeatable, 
analyst-independent ADP procedures had to be defined and documented and measures of crop identi- 
fication performance had to be determined. 

Each of the six 5x20-mile (8x32-km) sites was located in an overlap zone of ERTS so that 
coverage was available on two successive days on each 18-day ERTS . cycle . Of 72 potential, data 
sets from la'ce June through late September, only 15 were sufficiently cloud free for use in the 
analysis (See Table I) 1 

Periodic crop observations of fields used to train the classifiers were made by USDA/ASCS 
throughout the growing season.' These fields were found in 20 quarter sections (0.8x0. 8 km), each 
located randomly within a lx5-mi (1.6x8~km) strip in the site. Photointerpretation, of multidate 
aerial photography was successfully used to increase the size of the data base. The photointer- 
pretation data in 20 full sections (1.6x1. 6 ion), randomly selected throughout the site, were used 
to evaluate ERTS data classification accuracy in field centers. In addition, crop area proportion 
measurements were made from the; aerial photography and used to evaluate proportion estimates 
derived from pixel-by-pixel classification of ERTS data. The accuracy of photointerpretation was 
tested In 223 ASCS-visited fields which were not revealed to the photointerpreters. The results, 
summarized ir: Table II, were judged to be of sufficient accuracy to warrant use of the photo- 
interpreted field identifications; for .evaluation of the ERTS data processing results. 

Multiple passes of ERTS data were registered with an average root mean square (rms) error of 
less than one-half pixel, enabling multitemporal classifications of the data and eliminating the 
need to locate field and section coordinates in each ERTS scene. 

The need to maximize the number of pure pixels selected Horn the relatively small-sized 
fields present in several of the segnents made selection of field coordinates more difficult than 
expected. Manual methods were found to be inadequate for the job and a computer-aided method of 
transforming digitized photomap coordinates to ERTS line and column coordinates was used [1] . The 
use of the latter method is recommended for future projects requiring precise definition of ERTS 
data coordinates . 


A key task prior to the start of ERTS data classifications was to define and document data 
analysis procedures which were repeatable, easily followed, and yet incorporated the judgment and 
skill of experienced analysts. Although there was concern that crop identification performance 
might be reduced by restricting analyst decisions, it was necessary that variability due to 
analysts be minimized if meaningful conparisons of results were to be made. Limited tests with 
the LARS ADP procedures indicated that, for the CITARS data set, the procedures used produced 
results comparable to those obtainable using procedures with considerably more analyst interaction. 

To evaluate crop identification performance, three categories or classes of data were defined. 
The first two were the major crops, com and soybeans, while the third, called "other", included 
all other ground covers. Analysis of wheat recognition early in the year was attempted, bub the 
amount of wheat in the segments was too small and the reliability of its photointerpretation too 
low to support meaningful conclusions. 

An important accomplishment of CITARS was the use of quantitative measures of crop identifi- 
cation performance and statistical evaluation of results. The statistical analyses consisted of 
analyses of variance and blocked rank tests for conparisons Involving factors such as ADF pro- 
cedures, location, acquisition date, and use of preprocessing. Two variables, average conditional 
classification accuracy of "pure" field center pixels and the rms error of proportion estimates 
for entire sections, were used as measures of classification performance. Seetion-to-section 
variability was used in analyses of variance to determine if differences among ADP procedures, 
sements, times, etc. were significant. The analyses of variance revealed several significant 
differences, but the power of many of the significance tests was limited due to missing data, the 
amount of variability present, and the failure of the dependent variable to adequately describe 
performance for a section independently of the composition of that section, despite the use of a 
normalizing transformation. Continued use and development of these tools for remote sensing 
experiments are reconwiended . 


3. RESULTS AND DISCUSSION 

The statistical analyses provided a key to the quantitative assessment of remote sensing 
technology for crop identification, for both field center pixels and crop area estimation. Pre- 
vious results were confirmed in some instances, while in others unanticipated results led to 
reconsiderations and new Insights into certain aspects of the technology. This section summarizes 
the major results' from the CITARS experiments, as related to the six specific objectives which 
were expressed as questions regarding the effects of various factors on crop identification per- 
formance, , 

3.1 EFFECTS OF TIME DURING THE GROWING SEASON .- T- 

The time of ERTS data, acquisition during the growing season was found to be an Important 
factor influencing crop identification performance, because of the phenological development cycles 
or crop calendars of the major ground covers. Thf peak accuracy for field center classification 
was 75-80% correct in mid-August, as shown in Fir,.. 1. At this time, the variability within the 
major crops (corn and soybeans) was low and the 'amount of ground cover was high. 

The solid line in Fig. 1 represents the expected performance for the average of all single- 
date procedures, assuming no Interaction between the factors: site and time. The use of a non- 
interactive model for computing expected performance as a function of time was necessitated by the 
fact that only one site had ERTS data for more than two of the six time periods. The individual 
points marked on the graph represent actual performances by the various procedures. The varia- 
bility present is an indication that factors other than time also influenced field center classi- 
fication performance. : ■ j ■ L 

A similar expected performance time profile also was calculated for proportion estimates over 
the aggregation of whole test sections. Tills profile showed roughly the same rms proportion:, error 
for all time periods, except mid-July when the error was substantially greater. In mid-July, the 
variability among corn fields and among soybean fields was high, and the amount of ground cover 
was low. We do note, however, that variability in performance among procedures at any given time 
was much greater for proportion estimation than in the case of field center classification. 

3.2 EFFECTS OF SITE v ^ 

Missing data again hampered the analysis when conparisons were made between sites. Never- 
theless, proportion estimation accuracy was found to be much more site dependent than was field 
center classification accuracy, when expected responses were computed. The only major site: 
characteristics which were found to be correlated with proportion estimation accuracy Were average 
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field size and proportion of com and soybeans in the segment. As shown in Fig. 2, proportion 
estimation errors were smallest for the site with the largest average field size. Similarly, the 
proportion estimation error was found to be smallest for the site with the greatest percentage of 
corn and soybeans among its ground cover types. 

The correlation of field size and the accuracy of crop proportion estimates is attributed 
primarily to the decrease in the percent of pixels containing mixtures of crops as field size 
increases. In addition, it has been observed that large fields tend to be more uniform, and areas 
having larger fields have relatively fewer fields of "other" covers. The Influence of these 
factors on crop identification performance will be discussed further in Sec, 3.3* 

3.3 EFFECTS OF ADP PROCEDURE ON LOCAL RECOGNITION WITH SINGLE-TIME DATA 

EOD, ERIM, and LARS each defined a principal ADP procedure which was used for the major com- 
parisons of crop identification performance using data from single ERTS passes; alternate pro- 
cedures also were defined and tested by ERIM and LARS. In this section these procedures are com- 
pared for local recognition, that is, when the test data were located in the same site and ERTS 
pass 'as the training data. Nonlocal recognition is considered in Sec, 3.^. Identical training 
field, test field, and test section coordinates were used with all procedures. 

3.3.1 PRINCIPAL PROCEDURES, Major differences were found in results for the three principal 
ADP procedures. Local performance results with these procedures are summarized in 'Table III, 
where overall average performance figures are given, as well as the number of specific analysis 

of variance (ANOVA) comparisons for which significant differences were detected (out of a total 
of ten comparisons). The ERIM/SP1 procedure was best for field center classification, while 
LARS/SP1 was the most consistent for whole area proportion estimation and had the lowest average 
rms proportion error. These results Indicate that field center classification of pure pixels 
which has commonly been used to evaluate crop identification performance is not a reliable indi- 
cator of the accuracy of proportion estimates for whole areas. 

Primary differences between the ADP procedures lie in the training procedures and decision , 
rules used. Yet, there are some characteristics which they share that can contribute to the 
observed results . 

First, there is inherent bias in proportion estimates based on aggregated counts of maximum 
likelihood pixel-by-pixel classifications. Bias exists because the expected performance of a 
classifier depends on the true crop proportions present, as well as on its performance matrix for 
individual pixels. As can be seen in Table III, all. three principal procedures consistently 
underestimated the proportion of "other" in the test data. Furthermore, the expected rms error 
in proportion estimation was found to be correlated with tj s percent of other in a test site. 

Second, the whole areas included boundary pixels which contain mixtures of two or more ground 
covers. Mixture pixels were determined to be a major source of biased proportion estimates by a 
special . analysis , as well as by the fact that expected proportion errors tended to be largest in 
segjnents with the smallest average field size (Fig. 2) and, therefore, with the greatest number of 
mixture pixels, '■ . 

The three principal procedures tested differed in two ways. Both LARS/SP1 and E0D/SP1 used 
a clustering procedure to define training statistics (usually several classes for each major crop) 
and enployed a quadratic decision rule. ERIM/SP1, on the other hand, formed a single signature 
for each major crop, used a variable 'number of signatures for "other", and used a linear decision 
rule . . The differences in performance among the three procedures were determined, to be due to the 
method of training rather than the decision rule used since similar results (high ranking for 
field center recognition and low ranking for whole area proportion estimates) were Obtained for 
ERIM/SP2, a quadratic decision rule classifier, which used the same signatures as ERIM/5P1. It 
was observed that the disparity between rankings for the two types of performance for the ERIM 
procedure tended to be reduced or eliminated when within-corn and within-soybean variations were . 
smallest and the procedure selected greater numbers of other-class signatures , 

An attempt was made to correct for classifier bias in proportion estimates by using the per- 
formance matrix for field-center pixels, but the attempt was unsuccessful both on a section-by- . 
section and an aggregated segrent basis. 

3.3.2 ALTERNATE PROCEDURES. . A linear decision rule optimized on a class-pairwise basis was 
used in ERIM's principal procedure and a quadratic decision rule, similar to those of LARS/SP1 and 
EOD/SPl, in its alternate procedure. It was found that the accuracy of results with the linear 
rule were equal to or better than those with the quadratic rule using the same signatures. 

Resource constraints of CITARS did not permit similar conparisons with signature sets obtained by 
a different procedure, but such or similar conparisons are recontnended . 


Another CITARS result which, on the surface, seems surprising is the lack of Improvement of 
LARS/SP2 (non-equal major class prior probabilities) over LARS/KPl (equal prior probabilities). 
Theoretically , apart from boundary pixels, the Bayesian classifier should produce its minimum 
error rate when "correct" values for the frequency of occurrence of each spectral class are 
utilized as parameters in the classification rule. LARS/3P? included a procedure for estimating 
the prior probability of each spectral class based on existing agricultural statistics . For 
CITARS classifications the LAR3/SP2 procedure utilizing unequal prior probabilities did not pro- 
duce an improvement over assuming them to be equal. This is attributed in part to the fact that 
the agricultural statistics used were at a county level only, differing by as much as 20 percent 
from the true proportions in the test sections which were subsets of the county and not randomly 
located within it. Boundary pixels are another possible cause. We do not believe that use of 
prior probability information in the form of class weights should be discouraged, based solely on 
the CITARS analysis since it does not constitute a -definitive test. Instead, it is recommended 
that further tests be made to determine the sensitivity of maximum likelihood classifier to 
class weights . 

In other experiments LARS showed that significant differences in classification performance 
can be obtained with different training sets and that training set size alone does not determine 
the adequacy of a training set. These results and I'esults discussed earlier point out the 
dependence of crop identification performance on the development of training statistics. 

3 ,1) EFFECTS OP NONLOCAL RECOGNITION AND PREPROCESSING 

In nonlocal recognition, the training statistics are used to recognize data from a different 
location and/or a different ERTS pass . Such procedures are desirable and/or necessary in order 
to reduce the cost of obtaining ground identification information for training classifiers in 
operational applications. The effect of nonlocal recognition on performance with the single-time 
procedures was evaluated for 20 pairings of the 15 data sets. 

Comparisons of classification performance indicated that average field center performance for 
the three principal procedures in nonlocal recognition was reduced by 22 percent of that obtained 
locally . For whole area proportion estimates, the average rms error of nonlocal estimates was 
23 percent greater than that for local estimates. The degradation associated with nonlocal classi- 
fication performance was shown to be correlated (r = -0.77) with differences in atmospheric opti- 
cal depth (a measure of haze level) between the training and recognition segments. Other differ- 
ences present In the data sets were those of soil type, agricultural practice, crop maturity, 
scene composition, training data selection, and MBS scan angle, all of which can affect the 
representativeness of signatures. The results clearly indicate problems in successfully applying 
training statistics to different locations and/or ERTS passes. 

One way of extending the realm of applicability of signatures is to transform them radlomet- 
rically so they better represent observation conditions at recognition segments . Preprocessing 
with a mean level adjustment algorithm (ERIM/PSP1), a relatively simple preprocessing algorithm, 
was found to be of some help in inproving nonlocal recognition performance. Overall, the pre- 
processing procedure ranked above the three principal procedures for both whole area proportion 
estimation and field center classification ; this ranking was statistically significant for field 
centers but not for whole areas. Preprocessing cut roughly in half the degradation In mean field 
center performance and substantially reduced the correlation between optical depth differences 
and field center performance (from r = -0.77 to r - -0.28) , but was not consistent In its per- 
formance, especially for whole areas. 

The mixed results obtained in specific analyses, of variance indicate that differences in com- 
position of training and test areas also are important factors affecting nonlocal recognition. 
Additional research is required to improve upon the signature adjustment algorithm tested and to 
better account for spectral variability due to scene composition. A limited test of a more com- 
plex signature extension algorithm at ERIM, in an effort supplementary to CITARS, Indicated that 
improved results are possible. 

3.5 EFFECTS OF MULTI1BMP0RAL DATA 

One CITARS segment (Fayette) had several clear EftTS overpasses which were spatially regi- 
stered and then, analyzed and processed multitemporally with the E0D/MSP1 procedure, Significant 
increases in crop Identification performance were obtained, compared to the best single-date per- 
formance. Use of multltemporal data increased field center classification accuracy from 8l per- 
cent to 89 percent correct and halved the rms error in proportion estimation. These substantial 
improvements in performance, were obtained for this one segment by using basically the same data 
analysis procedures as for single-date data; nevertheless, new analysis procedures taking into 
account the increased complexity of multi temporal scenes will need to be researched and developed. 



Although use of multitemporal data requires a more complex data processing system (registration, 
increased data base size, and more complex data analysis procedures), the increased complexity 
may well be justified by increased performance. 

3.6 RELATION OF CROP AND .'SENSOR CHARACTERISTICS 

Two key factors influencing crop identification with remote sensor data are (1) the nature 
of the spectral variation ;rtnong aid within the classes to be identified and (2) the capability of 

the sensor to hko --'the spectral variation. An understanding of the relationship of these 

factors may help explain the levels of crop identification performance obtained in CITARS. In 
several instances it was found that accurate identification of corn, soybeans, aid "other" was not 
possible even when all the fields analysed were used to train the classifier. This may have been 
due to a lack of differences in the spectral characteristics of the three classes or to an 
Inability of the ERTS MSS to resolve and precisely measure the differences present. The latter 
is suspected to account for at least a part of the problem since crop classifications made during 
the 1971 Corn Blight Watch Experiment [ 3 ] using MSS data with more spectral bands, narrower bands, 
and greater sensitivity and dynamic range showed that these same cover types could be more accu- 
rately identified. Additional comparisons of ERTS and aircraft-acquired MSS or other high spectral 
resolution data such as will be available from the current IjACIK (Large Area Crop Inventory Experi- 
ment) field measurements project [*f] will be needed to verify this point. 


i|. CONCLUSIONS AND RECOMMENDATIONS 

il.l CONCLUSIONS 

CITARS has provided a quantitative assessment of 1973 -era technology for remote identification 
of major agricultural crops. The use of . quantitative measures of classification performance and 
statistical evaluations of the results have been important parts of the technology assessment. 

The major conclusions from the CITARS experiments are: 

1 . Crop identification performance for corn and soybeans varied throughout the growing 
season, with field center classification accuracy being maximum in late August. 

2. The probability of correct classification of field center pixels was not well corre- 
lated with and thus was not a reliable indicator of proportion estimation performance. 

3 . Proportion estimation accuracy was strongly correlated with both average field size 
and proportion of major crops in the segment, but field center classification accuracy 
was not . Boundary pixels containing two or more cover types were determined to be 
major contributors to the bias in proportion estimates. 

I!, The manner in which ground cover classes were selected and used to train the classi- 
. L fler strongly influenced the amount of bias in proportion estimates . 

5 . ' Probability of correct classification and proportion estimation accuracy both were 

decreased when training statistics developed for a different location or date were used . 

6 . A mean level adjustment algorithm for first order adjustments to training statistics 
used for nonlocal classifications increased the probability of correct classification 
of field center pixels, but did not improve proportion estimated' for whole areas. .. 

7 . The use of multidate data improved both proportion estimation.;aecuracy and probability 

of correct classification. | 

In addition it has been shown that relatively automatic data analysis procedures can be 
defined which produce repeatable results, are. suited for processing relatively large data volumes , 
anS incorporate, to a large degree, the judgtient and. expertise of experienced, analysts. 

i),2 RECOMMENDATIONS 

CITARS provides, valuable direction for future research and development of . remote sensing ; 
technology and guidelines for the design of operaWc-.al crop production survey systems utilizing 
remote sensing technology'. Recommendations from CHARS include: 

1 . Continued use and development of quantitative measures of crop identification per- 
formance and statistical evaluation of classification results . 


2. Continued development of improved methods for training classifiers, 

3. Research and development of methods to Inprove the accuracy of proportion estimates 
for whole areas. 

4 . Further tests to determine the sensitivity of maximum likelihood classifiers to 
the use of prior probability information and of the linear classifier to different 
signature sets . 

5. Additional research , development, and testing of two complementary approaches to 
nonlocal recognition, (a) more sophisticated preprocessing algorithms and (b) 
stratification of areas based on their similarity. with respepc to agricultural 
factors . 

6 . Development of data analysis procedures which account for the increased complexity 
of multitemporal data and take advantage of its potentially greater information 
content. 

7. Additional comparisons of ERTS and other multlspectral data sources to determine 
the adequacy of ERTS MSS in terms of the number, width and placement of its 
spectral bands, signal/noise ratio, sensitivity, dynamic range, and spatial 
resolution . 
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TABLE I. V 


ERTS DATA SETS ACCEPTED. FOR CITARS PROCESSING 


SITE. 



ERTS OVERPASS CYCLE ' 


JUNE 

26-30 

JULY 

14-18 

AUG 

1-5 

AUG 

19-23 

SEPT 

6-10 

SEPT 

24-28 

1. HUNTINGTON, IND. 


B 




' A 

2. SHELBY, IND. 





, B 

■ a 

3. WHITE, IND. 




B 

A 


4. LIVINGSTON, ILL, 


A 

A 




5 . FAYETTE, ILL. 

B 

A&B 


A 



6 . LEE, ILL. ■■ 


B 

i 

B 





Key;. A = First Pass of ERTS Coverage of Site 
B = Second Pass of ERTS Coverage 


TABLE II. COMPARISON OF CHOP TYPE ICEHTTFICATPOtlS MARK BY 
A5C0 AMD BY P1IOTOINTERPRETATIO}) 


PHOTOIiJiaWiED 


COVER TYPE 

STATISTIC 

ASC3 

TOTAL 

TOTAL 

CORRECT 

rxwsssioti 

ERROR 

CORN 

Number of Fields 

00 

51 

46 

5 


Percentage 

100.0 

1)2.0 

9P.0 

10.0 


Number of Acres 

1,181 • 

1,197 

l s :65 

32 


Percentage 

lOOi.O 

. 101.3 

98,6 

?.7 

SOYBEANS 

Number of Fields 

65 \ 

66 

61 

5 


Percentage 

100.0 

f! ”!-5 

°3.8 

7.7 


Number of Acres 

1,550 

v l.s'io 

1,523 

16 


Percentage 

ico.n 

99.3 

98.3 

1.1 

OTHER 

Number of Fields 

;v8 

m6 

99 

7 


Percentage 

100.0 

98.1 

91.7 

6,5 


Number of Acres 

879 

8?i| 

838 

36 


Percentage 

ioq.o 

99.1 

95.3 

'1,1 


TABIE III , COMPARISON OP PRINCIPAL PROCEDURES FOE LOCAL RECOGNITION 
A. MEAN CLASSIFICATION ACCURACY FOR FIELD CENTER PIXELS (15 CASES) 


CLASS 

LAR3/SP1 

ERIM/3P1 

EOD/SP1 

CORN 

0.66 

0.70 

0.62 

SOYBEANS ! 

0.59 

0.68 

0.61 

OTHER 

o.i r 

0.53 

0.46 

OVERALL, 

0,58 

0.64 

0.57 


o FOR SEVEN ANOVA CAPARISONS WHERE SIGNIFICANT DIFFERENCES WERE DETECTED 
FOR FIELD CENTER PERFORMANCE, EfilW/SPl BA! IKED FIRST IN SIX. 


B. MEAN PROPORTION ESTIMATION BIASES AND RMS ERROR FOR WHOLE AREAS (15 CASES) 


CLASS 

LARS/SPl 

ERIH/SP1 

E0D/SP1 

BIAS FOR! CORN 

O.063 

0.064 

0.025 

SOYBEANS 

0.033 

O.059 - : 

0.08l 

OTHER 

-0,096 

-0.124 0 ' 

-G.IO6 

RMS ERROR 

0.095 

0,150 

0.108 


0 FOR EIGHT ANOVA COMPARISONS WHERE SIGNIFICANT DIFFERENCES WERE DETECTED 
FOR PROPORTION ESTIMATION, LAR3/SP1 RANKED FIRST IN ONE AND SECOND IN 
SEVEN COMPARISONS, ERWSP1 FIRST IN THREE CASES, AND EOD/SP1 FIRST IN 
FOUR COMPARISONS. 
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ABSTRACT 

The CITARS task design, including objectives , analysis 
methodology and experimental procedures, is described and first 
results from this effort are presented. The extensive ground 
truth data set acquired for the CITARS task is described and 
discussed, in some detail. Results of the accuracy tests for 
the photo interpretative CITARS ground truth are given. Results 
of the assessment of the ERTS MSS data for cloud cover and 
electronic quality are presented . Sane results of the geometric 
correction and registration of the time sequential CITARS ERTS 
data are given. Finally, the field boundary selection problem 
is addressed and the results of the use of new technology for 
c boundary selection are presented . 


V 1, INTRODUCTION AND OBJECTIVES !,-■ 

In 1973* the Earth Observations Division (EOD) of the Johnson Space Center (JSC), the Environ- 
mental Research Institute of Michigan (ERIK), the Laboratory for the Application of Remote Sensing 
of Purdue University (Purdue/LARS ) , and the Agricultural Stabilization and Conservation Service 
(ASCS) of the U.S. Department of Agriculture (USDA), under took a joint task to quantify the crop 
identification performance, resulting from the remote identification of com, soybeans and wheat 
.using automatic data processing (AD?) techniques developed at ERIM, LARS, and EOD. These ADP tech- 
niques are automatic in the sense that subjective human interactions with the classification algorithms 
were minimized by the specification of the steps required for an analyst to convert a multispectral 
data tape to a classification result.. The crop identification performances resulting from several 
basic types of ADP techniques are to be. compared and examined for. significant differences: The 
multispectral data to be analyzed, consists of ERTS-1 data acquired over each of six 5 X 20 mile 
segments in Indiana and Illinois at six periods from early June through early September 1973- Crop 
identification and other, information was gathered by the. ASCS in each segment each 1 8 days coincident 
with an -ERTS overpass, ■ - 

The: ADP techniques are to be evaluated on this data set in two basic remote sensing situations : 

(1) Crop signatures for classifier training will be obtained within the same segment in which crops 
are . recognized by the classifier (local recognition). (2) Crop signatures for classifier training 
will be obtained from a different segnent in which crops are recognized (non-local recognition). 

Once the., crop identification performance is established for each of the ADP techniques for 
local and non-local recognition, differences in the performances of these techniques will be established 
for differences in geographic location, time of the year, etc. 


* Proceedings of the Ninth International Symposium on Remote Sensing of 
Environment, Ann Arbor, Michigan, April 15-19, 1974. 
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The CTTARS task was designed to cjuantitatively answer the following questions; 

o How does com, soybeans, and wheat identification vary with time during the growing 

season? 

o How does the crop identification performance (C1P) vary among different geographic 
locations having different soils, weather, management practices, crop distributions., and field 
sizes? i 

o Can statistics acquired from one time or location be used to identify crops at other 
locations and/or times? 

o How much variation in CIP is observed among different data analysis techniques? 
o Does use of multi-temporal data increase CIP? 

o Does use of radiometric preprocessing extend the use of training statistics and/or 
increase CIP? 

o How much variation In CIP results from varying the selection of training sets? 
o Does rotation or registration of ERTS data affect classification performance? 


2. ANALYSIS METHCDOLOGY 

To establish and compare the capabilities discussed In the Introduction, an experiment was 
designed to: (1) accurately estimate the crop identification performance (CIP) and (2) determine 

whether differences in the CIP' s for the various conditions are significant. 

Each of the CIP's are established as a result of a specific "treatment" combination; such a 
treatment combination is characterized by several factors ■ These factors are: (a) ADP technique; 

(b) data acquisition period; (c) location; and (d) training-recognition method . 

Each of these factors can, in turn, be characterized by levels. The levels of factor (a) are 
different ADP techniques to be assessed;, the levels of factor (b)* are the six data acquisition 
periods from June through September 19731 the levels of factor (c) are the six test sites in 
Indiana and Illinois. There are many possible levels in factor (d) but they can be characterized 
for the present by (l) local recognition and (2) non-local recognition. 

Each treatment combination will have an associated CIP which will be quantified in three ways : 
(l) a classification performance matrix from which the errors of omission and commission for 
"non-boundary" pixels can be determined and (2) a proportion classification error vector and 
(3) a proportion error vector corrected for bias. 

The classification performance matrix for. ''non-boundary" pixels will be established by 
comparing the ADP classification with the. ground and photo interpretive Identifications of about 
12,800 acres within each data segr.er.t. Test field boundaries will be established on the digital 
data. To Insure that. only non-boundary pixels are used in training and classification, the 
boundaries will be selected such that no agricultural field boundary elements or field inhomo- 
genieties are contained within the test field boundaries. The probability for correct classifica- 
tion for each of com, soybeans, wheat and "other" will be defined, for a particular test field 
set, as the frequency with which test field pixels of a particular class are correctly classified. 
The error of commission between class 1 and class j will be defined as the frequency with which 
an ADP identification of class i was determined from ground truth to actually have been a pixel 
from class 3 . For a four class data set this procedure will define alxA error matrix. 

The proportion classification error vector will be established by comparing the proportions 
of com, soybeans, wheat and other as determined from the ADP technique to those proportions 
determined from ground truth. To establish the ground truth, twenty agricultural quarter sections 
in each segment were visited each 18 days by ASCS personnel for crop type Identification. In 
addition, twenty additional agricultural sections (one mile square) were photo interpret to 
establish crop identification. 


#The levels in factor (b) will differ for the multitemporal ADP technique, i.e., if data from three 
passes are used for the analysis then there are ten possible ways to combine the six data acquisition 
periods . 


The proportion of each crop type in the sections within each segment were established by 
mensuration of the photography. This results will be compared to the proportions determined by 
the ADP techniques to determine the ADP proportion error vector. In addition, several methods 
have been proposed to correct the remote sensing estimates of the Grop proportions for bias . 

Each of these methods require an estimate of the bias, which is obtained by examining classifica- 
tion performance in fields or areas for which ground truth is available . These fields and areas 
will be cflled pilot fields or areas, to distinguish them .from the test fields where the crop 
identification performance is to be established. The methods proposed for bias correction and the 
method for pilot field selection are more fully discussed later in this section. 

Thus for each treatment a performance matrix and a proportion error vector can be estimated 
using the procedures described above. 

These data, once computed, form a basis for comparison of the performances of the techniques 
under the various conditions, lhese conparisons will be made using standard statistical tests. The 
primary statistical test to be applied is the analysis of variance. The objective of such a proce- 
dure will be to. test the hypothesis, that the classification performance for two or more different 
treatments (or combinations of treatments) are different. An example of a hypothesis to be tested is, 
"There are no significant differences in crop identification performance among test sites." To test 
this hypothesis , the ratio of variation among test sites is compared to the variation within test 
sites. This is referenced to as the calculated "F" which is the ratio of the treatment mean square 
(among) to the error mean square (within). If the calculated. F is greater than a tabulated F based 
on the known distribution of the variance ratio under the null hypothesis, then one would reject 
the null hypothesis and accept the alternate hypothesis that performances are different for different 
locations. Similar hypotheses can be formulated for each factor, The comparisons of interest to 
the FY7*I task discussed in the Introduction can be formulated Into hypotheses and tested in the 
manner described above. 

To Use analysis, of variance a measure of error must be available. This is obtained from . 
replication which is readily available in a factorial experiment. For example, a mathanatical 
model assumed is ; 

x ij = u + T i + e ij ,1 = 1, 2, ...» k, j =1, 2, .... n ••• 

which states that any observed value x,. Is equal to the. overall mean p for all populations, 
plus the deviation of the i-TH population mean p from the overall mean, plus a random 

deviation from the mean of the i-TH population . In other words , if p., is the mean of the i-TH 
population, then 

p = Sum of Uj/K 

■ e ij = x ij - w i = x lj “ u " T i . V;. : -t 

for this model it is assumed that 

1. p is mi- unknown parameter . 

2. The are unknown constants or parameters, 

2 ' 

3. The ey are normally and independently distributed with mean zero and variance cr . 

: ■ “ . ' 'O' r : v : ■ 

With estimates of the population means and variance o it is also possible to estimate the magnitude 
of treatment effects and to calculate confidence intervals . 


3. EXPERIMENTAL PROCEDURES 

3.1 TEST SITE SELECTION 

The test sites were chosen over as large a geographic area as possible, within the resources of 
the project, in order to Include a wide variety of conditions. Even in the Corn Belt there is a 
great deal of variation in soils, weather, cultural practices, crop distribution, etc. All of these 
factors are related to geographic location. The best measure of the effects of these factors, then, 
can be obtained by including as many test sites as possible over as large an area as possible. This 
also Increases the probability of obtaining cloud-free ERTS data. 
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Test sites were selected within the four overlap zones of the five ERTS passes over Indiana 
and Illinois . These areas shown in Figure 1 include sane of the different conditions which could 
he expected to be encountered in the Gom Belt. 

3.2 SELECTION OF SEGMENTS AND SECTIONS 

Segments , five by twenty miles , were chosen at randan within each of the six selected 
counties. These segnents were oriented such that the twenty mile length was oriented north- 
south. This segnent size was chosen to give a limited area for field visits and yet an 
adequately large area for a representative sanple of agriculture within a county. 

Within each segnent , 20 sections and 20 quarter sections were chosen at random in a manner 
such that the selected quarter sections .were spatially disjointed from the sections selected. 

3.3 FIELD OBSERVATIONS FOR CROP TYPE IDENTIFICATION 

The ASCS of the USDA visited , each 18 days , the 20 quarter sections within each segnent and 
examined each field In the quarter sections for crop type identification, other agricultural para- 
meters shown in Figure 2. Atmospheric optical depth (related to visibility) at several locations, 
using tripod mounted solar spectrophotometers provided by JSC and subjective assessments of cloud 
cover weather and haziness during the ERTS overpass were also recorded by ASCS personnel,' 

3.4 PHOTO INTERPRETATION FOR CROP TYPE IDENTIFICATION 

A more accurate estimate of the crop identification performance for each ADP technique can 
be obtained if a larger field sample is available from each segnent. Thus, the field observation 
data liras supplemented by photo interpretation of the 20 additional sections chosen in each segment . 

The photo interpretation effort used large scale color IR (scale) photography acquired at 
five times during the growing season, and large scale metric photography acquired at two times, 
to establish proportions of ground cover classes and other agricultural parameters within each of 
the 20 sections in each segment . 

A test procedure using ASCS visited quarter sections hidden in the photograph was devised to 
determine . the accuracy of the crop identifications so obtained. The photo. interpretation procedure 
was designed to obtain as accurate an identification as possible . When the PI test procedure indicated 
errors in the photo interpretation field Identifications, the effects of these errors on the estimates 
of the ADP crop identification performance were assessed, once the source and nature of the photo 
interpretative errors were ascertained. 

3.5 ERTS DATA PREPARATION 

This section addresses those procedures required to reformat the ERTS MSS data, to locate in 
the data the sections and quarter sections, and to choose the test and training fields within 
these sections.. These procedures have been designed to allow each institution to: use cannon 
training and test field boundaries and duplicate ERTS data tapes. Such a procedure was followed 
to permit more meaningful performance comparisons and to eliminate needless duplication of tasks 
and resources at each institution. ;; 

' ERTS buDc tapes were received at LARS for subsequent reformatting and field boundaries 
definition, if' 'Vv-.v':./ 

Of the two ERTS passes over each segment, the one acquired during minimum cloud cover was 
selected for local recognition. If cloud cover was equal for the two passes, the data acquired 
most temporally coincident with the ASCS field visit was chosen for local recognition processing. 

3.5.1 GEOMETRIC CORRECTION AND REGISTRATION 

ERTS data preparation for C1TARS has consisted of (1) geometric correction, (2) multi- 
temporal registration, and (3) section and field coordinate location. Geometric correction is 
perforated to facilitate accurate location of section and field coordinates. Registration of the 
data from two or more ERTS passes over the same scene is required for multitemporal data analysis 
procedures and to reduce the number of times which section and field coordinates had to be located. 

With registered data the desired coordinates need to be found only once and the sane coordinates 
are used for all data collected over the sane area. Field and section coordinates are, of 
course, required for classifying the ERTS data and evaluating the results. 

3.5.2 TEST, TRADING, AND PILOT FIELD SELECTION 

In addition to the training and test fields usually selected to train the classifier and to 
evaluate its performance, "pilot" fields were selected. The pilot fields will be used to determine 
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if a correction for bias to the classified crop proportion, resulting from classification errors, 
is feasible. Classification errors will be estimated in the pilot fields and based on these 
estimates, a correction will be applied to the test field classification results (See 3.8 FACTORIAL 
ANALYSES FOR PERFORMANCE COMPARISON for more details.) 

For those analyses which require pilot fields, all fields from one-half of the 20 photo 
interpreted sections will be used for pilot fields and the remaining ten sections for test fields . 

For those analyses Which require no pilot fields, all photo interpreted sections will be used as 
test fields. 

Training fields from the ASCS quarter sections will be used to train the classifiers. All 
fields large enough to be accurately located to the scanner imagery will be available for training. 

3,6 ADP TECHNIQUES FOR MSS DATA PROCESSING 

The basic ADP techniques will be grouped into three divisions: (1) "Standard" techniques, 

(2) preprocessing techniques , and (3) multi-temporal techniques. The term "standard" ADP technique 
Is used to mean either Gaussian maximum likelihood classifiers or a classifier employing a linear 
decision rule and classifies data which has not been radianetrically prepi’ocessed and has not been 
acquired multi-temporally . 

Each of these ADP techniques consists of a computer implemented software system and a method 
or procedure by which an analyst can convert multispectral data into ground cover class identifica- 
tion information on a pixel by pixel basis. 

Since the crop identification performance of ADP techniques can be sensitive to the manner in 
which the classifier is trained, the types of MSS data input (e.g. , preprocessed, multi-temporal, 
etc.) which spectral bands are used for recognition, etc. A quantitative evaluation and subsequent 
comparison of the crop ID performances of such techniques will be most meaningful if the procedures 
used to obtain the classification results are well defined and repeatable. 

Most of the existing procedures currently developed for the use of the very generalized analysis 
algorithms, require decisions on the part of the analyst which can significantly affect the classi- 
fication performance obtained . For the purposes of this assessment , the analyst factor will be 
minimized as much as possible to order to permit an evaluation of the automated techniques. 

A necessary requirement for a small variance to the classification repeatability of an ADP 
technique is that the procedure for using such a technique be sufficiently well defined so that an 
analyst can follow the procedure without deviation; thus, each of the ADP techniques evaluated in 
this task will be documented to detail, and the documented procedures will be rigidly adhered to. 

3.6.1 LARS ADP TECHNIQUES 

The analysis techniques to be" used by LARS utilize the LARSYS Version 3 multispectral data 
analysis system. Its theoretical basis ard details of the algorithm implementation are described 
in references 1 and 2, respectively. A complete description of the analysis procedures is con-, 
tained to reference 3- The procedures are designed to provide repeatable results, i.e. , variation 
due to analysts is minimized. Briefly, the analysis procedures consist of: 

1. Class Definition and Refinement . Four major classes, corn, soybeans, wheat (for . 
selected missions')' 'and ail "other"' ground" ^covers are defined, These major classes are divided 
into subclasses where spectral variability within a class is so great as to result in multimodal 
probability distributions for that class. Clustering is used to isolate the subclasses. For 
clustering all four PETS bands are used. A systematic method (see reference 3) which minimizes 
the total number of subclasses produced while ensuring, that multimodal class distributions are 
avoided is used for interpreting information on the separability of subclasses . 

Classification . Each data set is analyzed using two versions of the maximum likeli- 
hood classification algorithm. Gaussian probability density functions are assumed for both 
procedures . : The first classification method Is the maximum likelihood classification rule 
assuming equal prior probabilities for all classes and subclasses. This is the rule which has 
been in common usage for remote sensing data analysis for some time. 

. The second method uses, "class weights" proportional to the class y rior probabilities , This 
approach is more nearly optimal given that the Bayesian error criterion (minimum expected error) 
is preferred. Class -weights may be based on any reasonably reliable source of information. In ■: 
CITARS the weights are computed from county acreage estimates made by the USDA the previous year. 
Subclass weights are. simply the number of points to each subclass divided by the total number of 
■ points in the class. 
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3. Results Display arid Tabulation , The results of the classifications are displayed using 
a discriminate threshold of 0.1 1, This light threshold eliminates only those data points very much 
different from the major class characterizaticriSv-.Tbresholded points are counted in the ’'other" 
category. A computer program is used to generate results tabulations, in both printed and punch- 
card form, for training fields, test fields, and test sections, f ) 

3.6.2 ERIM ADP TECHNIQUES 

The digital data processing and analysis procedures defined by ERIM for use in the CITAR3 study 
reflect our concern for the calculations! efficiency of recognition processors and the need for extend- 
ing recognition signatures from training areas to other geographic locations and/or observation condi- 
tions, as well as the CTfARS requirement for minimizing the need for analyst judgnent. A brief summary 
of the procedures is as follows: 

3, 6*2,1 Training 

The training of the processor, that is, the establishment of class signatures for recognition, 
is a crucial step in multispeetral data processing. Although multimodal signatures are frequently 
employed, the use of one signature per major class was selected for CITARS processing because of 
simplicity, processing efficiency, and the fact that a combination of individual field signatures 
can result in a single signature that encompasses more of the variability of the class tha^ is 
represented by a multimodal signature. An objective, reproducible procedure, based on a X*" test, 
was devised to reject anomalous "outlier" fields before the formation of a combined signature , so 
as to develop signatures representative of healthy crops at a reasonable stage of maturity for the 
time of season. Signatures for classes other than the major ones are to be included only if they 
are found to be confused with the major crops on preliminary recognition runs over training data. 

3.6.2 .2 Recognition Without Preprocessing 

Two types of decision algorithms are being used, a linear rule and a more conventional quadratic 
rule. The linear decision rule was chosen because it requires substantially less computer time for 
recognition calculations, has been used successfully in many applications at ERIM, and has been found 
to provide ccmparable recognition accuracy in previous tests (reference 4)> Use of the quadratic 
rule will permit another , comprehensive comparison of the two rules . Both r.iie.3 apply a threshold 
test (0.001 probability of rejection) based on a quadratic calculation for .the signature of the 
"winning" class 5 points failing the test will be classified as being other than the major crops 
considered. . . ; . 

3. 6. 2. 3 Recognition With Signature Extension Preprocessing 

Changes In atmospheric and other local conditions can cause changes in the signal levels re- 
ceived at the scanner for different areas and at different times. The region of signature 
applicability can be extended beyond the region used for training by employing signature-extension 
preprocessing techniques. Non-local recognition denotes recognition performed on segments other 
than those from which signatures were extracted. Non-local recognition will be carried out once 
before and once after preprocessing corrections for signature extension have been applied (for 
both linear and quadratic decision rules). Several premising preprocessing methods have been 
developed (references 5 and 6) and are being tested on ERTS data at ERIM (reference 7). Only 
one method has been identified to date for use on the CITARS project — a mean-level adjustment 
procedure. The mean-level adjustment is derived from an average over diverse ground covers within 
the "local" signature extraction segment and a comparable average within the "non-local" segnenl 
to be classified. 

3.6.2. 4 Results Summarization 

The results obtained with each procedure will be summarized in a standardized form for 
subsequent analyses of variance. Separate summarizatlons will be made for field-center pixels 
and for entire test sections. 

3.6.3 EOD ADP TECHNIQUES 

EOD will evaluate two techniques. One technique for single pass data and another for multi- 
temporal MSS data. 

~ i 

For single pass data the EOD will utilize the IS0CE3 (reference 8) clustering algorithm 
Implemented at JSC to generate the class and subclass statistics and the Gaussian maximum likeli- 
hood classifier on the Earth Resources Interactive Processing System (ERIPS) (reference 9). V 

The training fields for com, soybeans and wheat will be submitted to independent runs on the 
Earth Resources Interactive Processing System (ERIPS) using the 1S0CLS implemented clustering routine 
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to generate class and, If necessary, subclass statistics, e.g. , com 1, com 2, com 3, etc. The 
training fields for "other" will then be submitted to the same clustering scheme to generate class 
and subclass statistics for all "other". 

The training fields, test fields and "test" sections will then be classified using the Gaussian 
maximum likelihood classification algorithm on ERIPS and the statistics as previously generated with 
the clustering process. 

For multi-temporal data ISOCLAS will be used to separate spectral classes . A linear combination 
of features will be selected using and EOD algorithm (reference 10) and classification will be similar 
to the uni-temporal technique. 

3.6 PERFORMANCE COMPARISONS 

The basic questions proposed in the objectives will be answered by a series of analyses of 
variance. There will be two basic quantities which will be used to characterize the crop identifica- 
tion performance of the ADP techniques. These are (1) e, . , the estimated probability of classifying 
a non-boundary pixel from class i as class j and (2) p^ - Pj_, the estimated proportion of class i 
minus the true proportion of class 1. 

Jn order to compute 
classes i and j must he 

will be chosen to exclude agricultural field boundaries within pixels and also to exclude known 
inhomogeneities in the field, such as flooded areas, etc. As will he discussed later, accomplishing 
this task was difficult and required the development of new technology. 

Since in an actual remote sensing situation, the classification error resulting from pixels 
containing agricultural field boundaries (boundary pixels) and the error resulting from field 
irihomogeneities may represent a large part of the error, some method is required to estimate these 
errors. The use Of e, to accomplish this was decided to be impractical because of the difficulty 
in locating the pixels 1 ttonta±ning field boundaries. Thus it was decided to use the proportion esti- 
mate to characterize this error, p^ will be computed for "pure test pixels" as well as for the 
agricultural sections and the differences in the resulting proportion error vectors will be used to 
estimate the error contribution resulting from boundary pixels and field inhomogeneities . 

In addition to the performance quantities discussed above, some attempt will be made to correct 
the proportion estimates for statistical bias which is expected to result fran misclassification. . 
Two methods have been proposed for accomplishing this and the corrected p . using each method 
(described below) will be compared to the p. as determined from the photo^interpretation to deter- 
mine if either method improves the proportion estimates . 

3.7 FACTORIAL ANALYSES FOR PERFORMANCE COMPARISONS 

For performance comparisons several dependent variables will be calculated for each of the 20 
test areas per segment. The quantities e^ will be estimated as discussed previously. 

The proportion estimates p^ will be computed in one of three ways: 

;1. ; : p t = n ± /N 

:/ &> % v n . 

i:®: 3. p ± = j 

where = number of pixels classified as i. 

■ N = total number of pixels in area to be classified. •••• ■■■;;■_ 

• . i; . B, = regression coefficient obtained by comparing n./N with the true proportion for 
V : pilot data ;'/■/ 

E = matrix of e^ ’s obtained from pilot data. 

n = vector of n^’s. 

Mote that methods two and three require the use of "pilot" data, i.e., additional ground truth used 
to obtain estimates of E or B^. 
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the e . , from- the ADP results, pixels which correspond to ground cover 
precisely located with respect to the ground truthed areas. Test fields 


Once a dependent variable is decided upon a typical analysis will consist of (i } obtaining col) 
.means of the dependent variable over various combinations of factors, (b) perforadngj >n analysis of 
variance . -• 

4 . FIRST RESULTS FRCM CITARS 

CITARS as designed and described above, was originally scheduled to begin in early June 1973 
and to be complete by October 1974 with a majority of the actual ERTS data processing complete by 
April 1, 1974; However, as of April 1, 1974, CITARS is approximately 9C days behind schedule. This 
slip resulted primarily from difficulties encountered in field boundary coordinate locatiun In the 
ERTS imagery (to be discussed momentarily). 

Complete as of April 1 is the acquisition of ground truth by ASCS, the aircraft acquisition of . 
large scale color IR photography , the interpretation of that photography for supplemental ground 
truth, data quality evaluation of the ERTS data, and the geometric correction and registration of 
that data.. To be completed are Test, Training and pilot field coordinate determination , ADP proces- 
sing of the data and the subsequent performance comparisons , The remainder of this paper will be 
devoted to a discussion of the completed portions of CITARS. 

4,1 DATA ACQUISITION rr 

Data acquisition consisted of three major efforts; the periodic visitation of each segment by 
ASCS personnel for crop type identification, the periodic acquisition of large scale aircraft photog- 
raphy and ERTS data acquisition. Each of these tasks have been completed and with some exceptions, 
each effort has provided data adequate for the accomplishment of the CITARS objectives. 

11.1.1 GROUND OBSERVATIONS 

ASCS personnel completed all field visits to all segments . In addition, they made additional 
visits in the Fayette County, Illinois segment to determine additional training and test fields for 
wheat which were required as a result of late and incomplete .aircraft coverage of Fayette County in 
early June, .. , 

Table 1 summarizes for each county the amount of acreage, the total number of fields by class, 
and the average field size for the fields in the 20 ASCS June identified quarter sections. In addi- 
tion to these and the other ag data discussed in section 3.3, ASCS personnel successfully used the 
solar photometers to record atmospheric optical depths on successive ERTS orbits over the segments.. 

For some of these segments, there are considerable differences in the atmospheric state from one day 
to. the next. Thus by training on a segnent and classifying it on the subsequent ERTS orbit, the 
effect of atmospheric differences on crop identification performance can be evaluated. 

4. 1.2 AERIAL PHOTOGRAPH'/ 

Aerial photography was used for field annotation, extension of ASCS ground truth via photo inter- 
pretation and mensuration of field acreage. For accurate photointerpretation, large scale color infra- 
red photos were specified. This photography was acquired each 18 days from June through October by 
the Bendix Queen-Aire Aircraft at 4 km altitude using a Fairchild 224 camera. For accurate mensuration 
of fields, data was acquired in July and August with a Zeiss metric camera flown in the ERIM C-46 at 
about 4 km altitude. Base photography for the annotation of ground truth was acquired with an RC-8 
camera carried at 18 km by the NASA RB57F. 

Camera problems, excessive cloud cover and incorrect flight lines rendered sane of the test and 
training sections in the Bendix photography unuseable. Some sections were also incompletely covered 
by the C-46. However, combining the photography from these two sources proved adequate for the CITARS 
requirements. 

4.1.3 PHOTOINTERPRETATION 

The photointerpretation team at EOD has completed the tasks of determining crop identification, 
areal proportion of each crop, row direction mid width and any field anomalies for each of twenty 
agricultural sections in all CITARS segments. This job wliich took about 6 months from the acquisition 
of the first photography was completed within Uiree weeks of the originally projected schedule. About 
18 man months was expended in the effort. The photointerpreters, using large scale color IK photog- 
raphy acquired 6 times during the growing season, identified 23 or 24 agricultural sections in each 
segment. The photo interpreters trained In 16 or 17 of the ASCS quarter sections. 'Dree or four of 
the remaining quarter sections were withheld frait the photointerpreterr, , but were Included in the Dili 
sections to be interpreted so that a comparison of the.ir results to the ASCS identifications could be 
made. The photointerpreters did not know which sections contained the tost- quarter sections. 
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Comparisons of the AHCS crop identifications to the I'l identifications have been completed in 
3 of the 6 CITARS segments. In these counties the percent agreement with the ASGS Identifications 
was 94$ for soybeans, 96$ for corn and 16$ for wheat. These are percentages of fields with a sample 
of 32 soybean fields, ?.J com fields and 6 wheat fields, Of the 2 soybean fields mlsldentifded as 
other* one field was planted 2 weeks late and the other one G weeks late, 'ihe corn field micciuasi- 
flcation may be a result of mistaken field labeling. This problem is as yet unresolved. The wheat 
problem is a different matter since the first aircraft photography of June 28 was acquired during 
wheat harvest. Thus the photointerpreters tod only nature (low IS reflectance) or harvested wheat 
imagery which proved inadequate for wheat recognition, 

4,1.4 ERTS-1 MULTISPEGTRAli SCANNER DATA 

The ERTS-1 satellite passed over each of the six test segments twice (on successive days) during 
each 18-day period. Since there were six periods of interest, from early June to early September 1973 
a total of 72 data sets were potentially available for processing and analysis . Cloud cover problems 
were identified by reference to the ERTS data catalog and visual inspection of imagery where available. 
Of the 72 possibilities , cloud cover was severe enough ( 20-30$) on 41 sets to cause their rejection 
outright, no data were collected for two sets, and several otters were eliminated for other reasons.. 

A total of 23 sets were selected for analysis and several of these wore found to have cloud problems 
upon detailed examination. Thus, roughly 6 g$ of the data sets were eliminated because of excessive 
cloud cover. Table 2 summarizes the cloud cover statistics. 

A majority of the selected data were of good quality. However a few problems were observed which 
are affecting data analysis procedures and/or results: (a) occasional erratic data throughout individ- 
ual scan lines or portions of lines, (b) detector-to-detector* differences among the mean values 
obtained from the six detector c'nannels that comprise each spectral band as averaged over a large 
sample of the data and («) differences in the variances observed from the detector channels over the 
same data sample. 

ERTS-1 data quality was assessed by seyeral different methods. First, visual inspection was 
made for each spectral band on a digital display to determine the presence' of any lines of bad data 
through one of the 5 x 20 mile segments. More than half (14) of the 23 seta had no bad lines, and 
the worst were one with 8, two with 19, and one with 40 had lines, Next, histograms and sample 
statistics (mean and standard deviation) were computed for samples of the data «*<- every line and 
every 30 th point for all cases, every line and every sixth point for many, and eyery line and eyery 
point for a few. These statistics were calculated separately for each detector in each spectral band.; 
unrotated ERTS data were utilized for these tests . 

One would expect some variation between values in the various detectors, because each is cali- 
brated separately . To evaluate the degree of similarity between these statistics, a mean, in , of 
the six detector means was calculated for each spectral band, as well as the sample standaxdteevia- 
tion, Sp, of the individual values from that overall mean. The ratio, Sy/uiy, was ccnputed for each 
data set and a histogram of these values is presented in Figure 3- All values lie below 3$, except 
for one Of 9.3$, which corresponds to the data set with 40 lines of bad data. No clear relationship 
could be found between the number of bad lines and Sy/mp values below. 3$. The number of good lines 
present was sufficient to mask effects of a few bad lines, and channel-to-ehannel variations existed 
in all data. 

Similarly, the detector value standard deviations were analysed and a histogram of sa/my is pre- 
sented in Figure 3. In this instance, the s/m values exhibit considerably more spread than they do 
for the detector means. Most values are <5$, but they scatter up to 24$, with an extreme of 87$ for 
the 4 0-bad-line case. Here egain, except for extreme values there was no apparent direct correlation 
between Sfj/nig and the number of bad lines present , . . . 

- The question now arises regarding the .analysis of data exhibiting problems associated 'With 
clouds, lines of bad data, and cliannel-to-charinel variations , Test section, test fields, and train- 
ing fields affected by clouds and/or bad lines were determined by inspection and eliminated from the 
analysis. Hie one data set with 40 bad lines is being analyzed in three bands only, since all bad s 
lines were in the some band. One could transform all data values to equalize the means and/or vari- 
ances in. each set of detector channels (omitting bad-line values) and perhaps effect some improvement 
in recognition results. However, geometric correction and spatial registration operations were being 
applied lo these data sets in parallel with the data quality evaluations, so it was decided to start 
again and carry out radiometric correction procedures only if poor recognition performance were 
obtained and appeared to be attributable to such differences. 


*De lector is used here to denote the entire signal channel from individual detector element to CCT. 
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■ if. . 

4.2 ERTS MSS DATA PREPARATION *!■ " 

Jf 

4.2.1 GEOMETRIC CORRECTION 

The digital form of the ERTS data (CCT’s) contains several geometric distortions . These 
distortions include : scale differential! altitude and attitude variations , earth rotation skew, 

orbit velocity change, scan tine skew, nonlinear scan sweep, sears angle error, and frame rotation. 

The major errors are the scale and skew errors. Also, rotation to North-orientation is highly 
desirable. A tv7o-step process, developed by LARS, to geometrically correct ERTS data over small 
areas has been applied to all data for CJTAR3 (reference 11). 

Briefly, the procedure uses four linear transformations to correct or adjust for horizontal 
and vertical scale differences, rotation, skew due to earth rotation and output scale factor. The 
process assigns radiance values in a rescaled, rotated, and deskewed coordinate system using data 
from the existing grid, i,e., the raw EHTS data. Because a fixed grid output device is used. (i.e. s 
line printer or digital display screen) some interpolation is required to produce new samples . The 
point nearest the desired sample location is used to represent the value at the desired location 
("Nearest Neighbor Rule"). 

The procedures are fully described in reference 11. The output form used for CTTARS is such 
that when the data is printed on an 8 line per inch, 10 column per inch computer line printer the 
resulting scale is approximately 1:24,000 and the image is North-oriented (Figure 4). Comparisons 
made using topographic maps indicate about a 1 to ?.% scale error, 

4.2.2 SPATIAL REGISTRATION OF TIME SEQUENTIAL ERTS DATA ■ 

Registration of multiple images of the same scene is accomplished through use of the LARS 
image registration system described in reference 12. The overlay processing operation consists of 
two basic operations: (1) image correlation and (2) overlay transformation. Many factors exist.;;: 

which prevent exact overlay of the images, thus this operation is approximate. Two major error?/ 7 ' 
are; (1) It Is unlikely that the samples from one time were imaged from exactly the same spot . ds 
samples from a later satellite pass; thus, in general, no data exists which exactly over lays vt’er 
both times even if no other errors were present; and (2) Due to changes in the scene and other 
"noise" sources the two images cannot be exactly correlated or matched. The overlay procedure used 
consists of the following: 

1. Initial checkpoints or matching points are manually selected in the two images to Le 
overlayed using a digital display screen. 

2. A two dimensional least squares quadratic polynomial is generated to represent the 
difference in position of points in the two Images. 

3, A block image cross correlator Is employed to find the remaining image displacement 
at the nodes of a uniform grid using the approximate overlay polynomial generated in inference if . 

4, A new overlay polynomial is generated from the correlator produced set of checkpoints 
and used to actually overlay the images. The two images are combined onto one data tape and a new 
data set is formed having M+N channels where M Is the number of channels from image A and Il- 
ls the number of channels from image B. 

5- The overlay data tape is inspected to check image quality ;md overlay quality, precise 
evaluation 'of overlay accuracy is difficult. A measure of registration error is obtained from the 
residual of the least squares polynomial; this statistic averages 0.5 of an image sample, EM3 (Table 3) 
However, crop Identification performance obtained when using field coordinates selected from a base 
period will be compared to performance when using coordinates located in data which has not been 
registered . 

4.7.3 SECTION : AND FIELD COORDINATE LOCATION 

Locating section and field coordinates in the ERTS data has been a major task preparatory to 
classifying the ERTS data. This task was first attempted using a manual method for location of 
fields in ERTS data displayed as single-band gray-scale line printer maps (reference 13). This 
required that field boundaries be easily distinguished in the imagery. In cases where there was 
minimal spectral contrast among crop fields, n on-supervised classifications have been performed to 
produce an enhanced image. Whether using gray-scale or computer enhanced images, reasonably 
large fields are required to assure that pixels are selected from within the field boundaries. 

With the CITARS data, there was little contrast among fields of interest, since the first data 
was collected early in the growing season (June 8-12). For instance, at this time of year com and 
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soybeans were only a few inches tall and the spectral response was primarily from the soil. And, 
roads were not as visible in the imagery as they generally are in data collected later in the season. 
Also, many fields were small {<20 acres). Therefore, procedures for accurately locating fields, 
when individual fields could not be clearly seen in the imagery, were required to meet project 
requirements. 

To Improve the accuracy of the manual location method, ERTS images were geometrically 
corrected and and rescaled to produce a nominal 1:2*1,000 scale map on a line printer (reference 11). 
This product alone made the location of fields more precise and "ore rapid than it would have been 
on Uncorrected data. Photo overlays were prepared with section and field boundaries outlined. The 
initial overlays, made from photography enlarged to a nominal scale of 1:2*4,000, were helpful, but not 
completely satisfactory due to distortions in the photo. Following this rectified photographs were ii 
produced at a scale of 1:2*1,000. This product could be manually overlaid to the line printer maps of 
the ERTS data. 

After manually locating all field and section coordinates in the ERTS data the precision was 
still not adequate to meet the requirement of a maximum error of <ne pixel. Therefore, a previously 
developed, computer-assisted procedure was employed by ERXM to locate section corners and define ERTS ' 
data coordinates for sections (reference 1*4 ) , A map transformation from Earth coordinates on a 
rectified aerial photograph to ERTS data coordinates was calculated for each segment using roughly 
30 control points for each calculation. The control points were located visually in the rotated 
and geometrically corrected ERTS data and by coordinate digitisation on the photograph. A map 
transformaton then was computed by the method of least squares * ERTS coordinates of the few control 
points with large residuals (>1 pixel) were checked ard modified or deleted, as appropriate, and 
the transformation was recomputed. Next, the transformation was applied to all section corners of 
interest (whose locations on the photograph had been digitized at the same time as the control 
points) to find their fractional line and column coordinates In the ERTS data. Final standard errors 
of estimate (for control points) were less than 0.5 and typically between 0.2 and Q.*4 ERTS pixels, 
i.e., 15 to 30 meters on the ground. The RMS error in digitizing the location of the individual 
points was on the order of three meters on the ground (errors of roughly 0.005 inch or less on a 
photograph at a scale of 1:2*4,000). 

These section corner coordinates (calculated in fractional ERTS line and column coordinates) 
were used to locate field boundaries of individual fields within the sections. A major advantage 
of the procedure is that it preserves the relative positions of all points considered with an 
accuracy that cannot be matched manually. Another feature of the ERIM procedure was utilized to 
generate ERTS data coordinates for each outlined section. All pixels whose centers fell inside 
lines connecting the vertices (again, located by fractional coordinates) were automatically 
included on coordinate definition cards. 

*4.3 ADP ANALYSIS OF THE ERTS DATA 

As a result of the difficulties encountered with the field boundary selection problem, the 
ADP analyses of the ERTS data and the subsequent performance comparisons as specified by the CITAHS 
design plan, have not been completed. However, the preliminary analyses to date merit some brief 
discussion. : . 

The most significant result to date, is the amount of training data acquired from the 20 quarter 
sections in each of the 5 x 20 mile segnents. 

To insure that the pattern classifiers were being trained only on "pure" and correctly 
identified ground classes, the CTEARS task design specified that training data come from the 
quarter sections visited by ASCS personnel and that no pixel (ERTS data resolution element) 
which contained a boundary between different ground classes be used in the computation of training 
statistics. During the CITARS design phase, the anount of ground truth required for classifier 
training was estimated by assuming that ten times the. number of channels used for classification 
would be required to train the classifier for each ground cover class. Thus based on 20 ground 
cover classes, and four channels, 800 "pure" or non-boundary pixels would be required for training. 
Other rough calculations (reference 15) indicated that no snore than about one half of all acquired 
pixels would contain agricultural boundaries based on a preliminary estimate of a 20 acre average 
field size for Indiana and Illinois, Thus for ERTS pixels of 1.1 acres in size, roughly l600 acres 
would be required to obtain the 800 "pure" training pixels. In addition, an equal additional 
amount of training data was required to form replicate training sets to determine the effect of 
training set selection on classifier performance. Thus ASCS was requested to visit and identify 
twenty 1 60 acre quarter sections to obtain 3200 acres for training purposes. 

"It' is of interest to see how this, design worked out in practice. In each, of the CITARS seg- 
ments, training field boundaries lave been selected and final boundary adjustments are nearing . 
completion. Based on the number* of training pixels selected to date, Figure 5 plots the percent 
of the training acres actually selected as "pure" training pixels versus the average field size 
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in the ASCS quarter sections. Except for the anomoly in Livingston County (yet to be explained) 
the shape of this curve is as one would expect, However , the percent usable pixels are much 
smaller than the early estimates of 50?. This is a result of a subsequent design change in C1TARC 
to include a row of "guard" pixels between the agricultural field boundary and the training field 
boundary. This was done to increase the probability that only non-boundary pixels were chosen for 
training, but prevented the selection of pixels in many fields, especially ones less than ten acres 
(20-40? of the fields ) and larger but narrow fields . 

5, SUMMARY 

The CITARS task was designed as carefully as possible to Insure an objective and quantitative 
assessment of the crop Identification performance of currently available classification algorithms 
and procedures . 

The ADP procedures were written to minimize the amount of subjective human interaction. This 
was done to permit a quantitative and repeatable evaluation of classification techniques which could 
be automated for operational implementation. 

For the resources available, a data set was designed to penult an objective evaluation of these 
techniques over a wide as possible range of agricultural and. climatological conditions . An extensive 
data set has been collected and the utmost care has gone into the preparation of this data set for 
ADP crop identification performance evaluation, 

Based on the data set acquired and the stated objectives of CITARS, a factorial analysis has been 
desigied to obtain the maximum amount of reliable information regarding classifier performance. 

At this point in the progress of the CITARS tack, the most major problem encountered was the 
selection of field boundaries in the ERTS data, Thio problem resulted from the requirement that 
no pixels which contained boundaries between different agricultural classes were included in the 
training or test data. This problem had to be resolved through the implementation of recently 
developed technology and has resulted in a 90 day delay. 

At this point, the combination of ASCS field visits with interpretation of low altitude 
temporally acquired photography appears to be a relatively cost effective and accurate method 
for as.sumbling a large ground truth set with stringent design requirements. 

Of the 72 possible ERTS acquired data sets roughly 60? of the data sets were unusable' as a 
result of excessive cloud cover. Of the remaining sets the electronic data quality was acceptable 
for processing. 
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ASCS SUPPORT FOR CROP ID TECHNOLOGY ASSESSMENT 
GROUND OBSERVATIONS SUMMARY 


COUNTY. 


DATE. 


.OBSERVER. 


FIELD 

ID 

1 

ACRE'S' 

(EST.I 

2 

CROP 

3 

HEIGHT 

4 

STAGE OF 
MATURITY 
5 

ROW 

GROUND 

COVER 

8 

OBSERVATIONS 

9 

DIRECTION 
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WIDTH 
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ROW DIRECTION KEY-COLUMN 6 

0 - NO ROWS 

1 * N - S 

2 - E- W 

3 - NW - SE 
A - SW-NE 
5 - CONTOUR 


GROUND COVER KEY-COLUMN 8 COMMENTS:, 
o - o-5y„ 

X - 5-20’/. — — - 

2-20-50 ■/. 

3 - 50-80*/. 

A - 80-1 ODy* 


OBSERVATION KEY 
0 - NO COMMENT 

EARLY IN CYCLE: 

1- BARE SOIL 

2- FRESHLY CULTIVATED 

3- NEWLY SEEDED 

4- VOLUNTEER 

5- REP LANTED 

6- REGROWTH 

HOMOGENIETY: 

11- THIN STAND 

12- DROWNED SPOTS 

13- SKIP ROW PATTERN 

14- STRIP FARMING 

LATE IN CYCLE: 


21- HARVESTED 

22- PARTHARVESTED 

23- GRAZING 
,24-DEFOLIATED 

25- WIND ROWED 

26- C HOPPED FOR SILAGE 

27- TILLED AFTER HARVEST 

STRESS FACTORS: 


31- DROUGHT damage 

32- EXTREMELY WEEDY 

33- PLANT DISEASE 

34- NUTRIENT DEFICIENCY 

35- HAIL DAMAGE 

36- LODGING 

37- INSECT DAMAGE 

SURFACE MOISTURE: 


41- DRY 

42- MOIST 

43- WET 

44- STANDING WATER 
‘ 45-IRRIGATING 

' OTHER: 

. 51 _ 


FIGURE 2. ASCS GROM) OBSERVATION SUMMARY FORM FOR CROP IDKCIFICATION AND CONDITION 
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FIGURE 3. VARIABILITY OBSERVED BETWEEN THE SIX DETECTOR CHANNELS 
IN SINGLE BANDS OF ERTS DATA 



Comparison of original and geometrically corrected and rotated ERTS-1 
MSS digital inane ry . Upper image is the original. Lower image is skew 
and scale corrected and rotated to North. Scale is such that when this 
data is printed on an 8 line per inch, 10 column per inch computer lire 
printer, the resulting scale will be approximately 1:2'* ,000 . 
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T ABLE 1. SUMARY OF ASCS IDENTIFIED QUARTER SECTIONS 
COUNTY CORN SOY WHEAT OTHER TOTAL 



ACRES 


iMKiMHM 


620 

3550 

Lee 

NO. FIELDS 


J M 


34 

160 


AVG. SIZE 



ill 

18.2 

22.1 



1239 

1073 

39 

569 

2969 

Livingston 

NO. FIELDS 

33 

27 

2 

33 

87 


AVG. SIZE 

37.5 

39.7 

19.5 

17.2 

34.1 


ACRES 


2B7 

4l7r 

1358 

~H33 

Fayette 

NO. FIELDS 

37 

11 

26 

92 

217 


AVG. SIZE 

19.8 

26.0 

16.0 

14.7 

14. 7 


Ill'll III— Will III ■MIiIMIiH 

IH35 

510 

38 

954 

3753 

White 

NO. FIELDS 

42 

13 

2 

41 

146 


AVG. SIZE 

^3.7 

39.2 

19.0 

23.3 




IB88 

54o 

323 

753 

364a 

Shelby 

NO. FIELDS 

71 

24 

15 

61 . 

189 


AVG. SIZE 

26.5 

22.5 

21.5 

12,3 

19.3 


Ml 


5l?T 

63 

986 


Huntington 

NO. FIELDS 

39 

25 

6 

54 

148 


AVG. SIZE 

21.2 

24.7 

10.4 

18.3 . 

18,6 
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Table 2 . Percent Clout! Cover over CITARS Test Sites During CRTS Passes . 

Cate 


Segment 

Pass* 

June 

8-12 

June 

26-30 

July 

14-18 

August 

1-5 

August 

19-23 

September 

6-10 

Huntington Co. , 

1 

A 

X 

X 

X 

X 

X 

Indiana 

2 

A 

X 

A 

X 

A 

X 

Shelby Co. , 

1 

A 

X 

B 

X 

C 

B 

Indiana 

2 

X 

X 

X 

.■ X 

X 

X 

White Co. , 

1 

A 

X 

X 

X 

X 

X 

Indiana 

2 

X 

X 

X 

X 

■ A 

c 

Livingston Co. , 

1 

A 

X 

B . ■ 

B 

C 

X 

Illinois 

2 

X 

c 

X 

x - 

X 

X 

Fayette Co . , 

1 

A 

X 

A 

X 

A 

X 

Illinois 

2 

A 

A 

A 

X 

X 

X 

Lee Co. , 

1 

X 

X 

B 

X 

X 

X 

Illinois 

2 

X 

x ■ ■ . ■ 

A 

A 

X 

X 


Percent Cloud Cover ^Segments are located in overlap areas 

A 0 _ 5 between ERTS passes on successive days 

B 6-15 
C 16 - 30 
X 31 - 100 


Table 3. 

RMS Error of 
ERTS Data. 

Spatially Registered Multi -temporal 

Segment 

Period 

Number 

Checkpoints 

RMS 

Lines 

Error 

Columns 

Huntington 

1 

30 

.44 

.36 

3 

27 

. 31 

.39 


7 

51 

.30 

.43 

Shelby 

3 

23 

.63 

.38 

5 

9 

.27 

.27 


6 

43 

.30 

.47 


7 

59 

.31 

.43 

White 

5 

61 

.32 

.39 


6 

16 

.28 

.14 

Livingston 

2 

32 

.31 

.44 


3 

9 

.44 

.37 


4 

9 

.16 

.87 

Fayette 

1 

41 

.39 

.33 

2 

52 

.37 

.29 


3 

53 

.44 

.34 


5 

19 

.57 

.39 

Lee 

3 

100 

.34 

.58 


4 

■ 84 ■ 

.32 ;■ 

.41 
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